Papers with Token pruning
Reducing Token Redundancy in LVLMs: A Systematic Review of Token Pruning Methods (2026.acl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) excel at visual understanding but face severe computational bottlenecks when processing high-resolution images and long videos due to massive visual token counts. |
| Approach: | They propose a taxonomy categorizing methods into vision-side, LLM-side and hybrid paradigms and analyze token selection mechanisms and pruning strategy. |
| Outcome: | The proposed method selectively removes less informative tokens while maintaining performance. |
Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained transformers suffer from a computationally expensive self-attention mechanism that interacts with all tokens, including those unfavorable to classification performance. |
| Approach: | They propose to integrate token pruning and token combining strategies to improve model performance and reduce computational demands. |
| Outcome: | Experiments with various datasets show that the proposed model performs better than baseline models, with the best improvement over the existing model. |
Walk and Read Less: Improving the Efficiency of Vision-and-Language Navigation via Tuning-Free Multimodal Token Pruning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large models achieve strong performance on Vision-and-Language Navigation tasks, but are costly to run in resource-limited environments. |
| Approach: | They propose a method to prune large models to minimize information loss . they use navigation-specific traits to filter the model into foreground and background . |
| Outcome: | The proposed method outperforms previous work on standard VLN benchmarks while saving 50% FLOPS. |
TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing token pruning methods rely on costly calibration or suboptimal importance metrics, leading to redundant retained tokens. |
| Approach: | They propose a token pruning strategy that preserves cross-modal alignment and informational diversity. |
| Outcome: | The proposed method maintains strong performance while reducing tokens by 88.9% on two models. |